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'^he original requirement for evaluation 'in Title I of* 
the Eleflientaty and Secondary Education Act of 1965 was unprecedented. 
However, th^ wording 'and intent of the reguir^ement was purposely-^lgf t 
vague becaus'^ of the conflict between the schoolman's concern 'over 
federal interference and Congress' demands for feedback that coul^ 
lead to efficient spending of the funds. Evaluation can.be a useful ' 
management tool for the practitioner .of Title I -because -it explains 
w-hether , objectives are actually being met'" and because it gives needed 
data on whether" to continue, modify, or terminate a project. Such 
considerations must be balanced with , educators * tendencies to \ ^ 
maintain the status quo. Some recommended characteristics of . ^ 

'9valura"tion 'reporrs should be; accentuate the 'positive- in reporting 
^fsulrs; use of layman's language, .especially - in a summary section; 
reevaluate essentials every year and nonessentials*.sp.oradically; be 

• sure that 'feach evaluation reports on some common strand of 
effectiveness, such.as^pupil achievement in reading and mathematics; 
and use longitudinal studies for information purposes only, not to 
evaluate the overall effectiveness of Title I. Finally, the 
evaluation should not be able to determine which services ^ny child 
is ,to. receive. ' (Authbf/RC) 
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A PRACTITIONER !S VIEW: ■ 
EVALUATING THE EVALUATION OF ESEA TITLE I ' ■ 

Historical Perspective . ' . . • 

' The expenditure of the funds authorized by Title I of the ETementary 
and Secondary Education Act of 1965 has now proceeded for a decade. The 
authorizing legislation for these expend i tu r,es , was unique invthe history of 
education in oUr nation. Never before, had Congress authorized such large 
sums for elementary and-^^^s^condary education; never before had anyone 
au.thori2ed ,bil l,ion,5 of dol lars' iFor the .education of the poor; never before 
had such restrictions been placed on the expenditure of funds for education 
by the federal- ^ove^rnment . ' ■ ' 

6otne nembers of Congress had grave reservations about allowing educators 
any type'of discretion in spending funds for th'e poor. Some f6U that educa- 
t;ion of the poor was a low priority among many educators and that they\'/ould . 
divert the funds to other uses. Robert Kennedy", probably more than anyone 
e'lse; wanted to assU»*e. that such massive funds were in fact used to .develop 
special and suecess-fulyprograms for the disadvantaged.- As a result, he de- 
manded some type of feedback- wMch* would -provide both legislators and parents 
with the Jnformation they needed to assure, that Title I dollars were being ■' 
used to helD poor children. Mr. Kennedy felt that the reports produced wou]d 
lead to improved local management practices and to better oppartunities for 
efficient federal direction to education. . ■ ^ 

-•Jhe Elementary ani Secondary-Education Act iof 1965 represented the firS'l\ 
m^jor piece of federal legis.tation concerning' edd^ation that mandated evalu- 
ation. The evaluation requihcment of tn>5 act was unprecedented,. The funds 
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were designed tomee^he /pecial educ^i^ional n^eds of educationally dtsad- 

~~ , ■ / ■ ■ 

vantaged children, and i|r' was felt that the evaluation requirements ^ 

/ ' ' , ~ " 

provi^^e up-to-date' infc^ation about the outcomes of Title I efforts with, the 

/expectation that ioca^i districts could adjust a;id refonn their^l^i-aalces in 
educating poor childreVi. In addition, it was expected that the resulting 
influx or well-organi?ed information would lead to efficient federal manage- 
ment of education programs. The key concept was that the availability of 
evaluation reports woufd lead to/fhange and reform in current practices! 
Such v/as the dream! ' - * " ,\ 

.But from the beginning the hopes for evaluation ran into oppositioTi. 
Both local and state educators feared federal interference in what was con.- ~ ' 
sidered a Slate right. Th6y were-?ipDrehens ive that ft would allbw-compar- 
•isons between locil school di'strkts .that' were unequal in the resources 
already .ava.il able to them. . They reasoned that- evaluation practices would be - 
costly and that^the measure-s to"- be used, especially achievement scores, were 
inappropriate- in that they measured' only one aspect of the learning situation. 
As a result of the conflict between the schoolmen's concerns and Robert " 
Kennedy's demands, the wording and th^'intent of the requirement for evalu- 
ation v/as purposely left; vague-'when ESEA was finally written. 

• The national evaluations t'hat hav.e resulted have- failed to meet the' "- 

• ^ - ' ■ ~ - - » • - 

expectations of the early theoriets and have, to a .great-deirccon the na- 
tiona^ scene, failed to mea'stire up to the needs of federal program off ic'l a Is. 
Both USOE officials 'and national evaluators have been una Dle^^c^tfie- past ' 
decade to, produce _ evaluations that are indicative of-^'io progress orTacl 
it that has been made by fe'd.uca^tional ly ,disadvantare bupils as a result of ' 



the expendittire of ESEA Title I funds. * • 

Evaluation At The Local Level 

ESEA Title I as it was originally conceived and as' it continues to oper- ' - 
ate today has its operat.ional authority and base in the thousands of local 
educational agencies of our nation. Needs assessments are done 1-ocally, pro- 
grams are formulated locally, and it follows that evaluation designs areTo- 
cally devised aod executed. It is this situs at the local level which has ' 
been one of the main reasons ;for the failure of national efforts to evaluate' 
the over-all effect of TitleVl. However, even at the local level there have ' 
been certai-n problgms. * 

Originally, local districts had no incentives to collect or report data-, 
anrofficials at the federal level had diffkulty in mandating "that it be done. 
In addition, local districts tended to be defensive about their achievement ' 
scores and about the long-term effects of Title 'l projects on pupils. Educa- 
tional policy-makers displayed a basic resistance to any tType of assessment and- 
/showed indifference and disinterest in any "eva luations that were done. Finally, 
'local districts each set up different goals and objectives/and different treat- 
hients, and it became difficult-to; prescribe all-encompassing objecti vef and 
strat^ies that were capable of precise measurement. In spite of these restric- 
tions and problems many loCaT distHcts mS'de and continue to make honest efforts 
to evaluate Title I projects in their school districts..- - \ ■ ^ ^ 

The Meeds Of The Pres ent ^ • ' ~ 

However, the facV remains that legislator\, educators, and the cornunity, • 
at large want to know what has been done with fifteen billion dollars in the 



last decade. It is indefensible to allow billions of dollars to disappear 
into the nation's educational system "without a trace. It is the responsibil 
^ty of the evaluation community of offer answers as to what has been doft'e 
'. wit^ these fynds over the years and to develop systems that can coordinate 
the myriad of results that TjtTe I" funds have produced. If is a concurrent 

resporjsibility of local federal program directors to understand -the neces- 

Y 

sUy for some type of consistency^^n eva-luat.ion and cooperate in its develop 
ment. Evaluation as a process Continues .to grow in importance as more and 
more attention must be directed at assessing and 'improving- results . 

My practical experience with the evaluation of ESEA Title I has been i 
the Cleveland Publ ic. School system. I have become convinced that over the. 
years evaluation of Title I in Cleveland has shown marked improvement and 
has become sophisticated to the point where the annual reports represent 
superior work. The comments and recommendations which follow are based o 
various aspects of the evaluations Which I receive annually. Practically 
all these items are alJ-eady a,n integral part of Cleveland's evaluation 
reports of Title I. 

' • . « - • V 

Evalua'tion' As /V Management Tool 

' My owi) views and perspectives of .evaluation have changed in the nine 
years th^t I Ij^ve worked with ESEA Title 'I and with other s|\acial compen- 
satory fuT^.. Originally, evaluation merely gave-.feedback on\J\/hether 
pa/ticular^ programs at the local Idve^ were being implen-nted as designed 
v^and whether they were effective in terms of the^asons fon, which they 
were initiated. However, as time progressed evalur'- ^ came to serve a 



a variety of functionsv and became^^aSiseful managernefrKtool 

First of an, evaluation allowed me to ascertain wheth^r<the various 
objectives of the project were being met. The vn6wpoint of a persor^&ut^^ 
the fohnal operation of ,the project was given as to whether the process v/as^ 
preceding a^^o^riginally conceived and* whether this process led to predicted 
changes in the pupils served. Secondly, evaluation gave me results which 
could be used to justify either the continuation, the termination, or the 
redesign of particular projects. Where pupils achieved the objectives of 
the pri)iect, especially tn terms of achievement objectives which had been 
set and in terns of the academic areas 'of reading and mathematics which' 
"continue to be high priority areas of'school systems-,' evaluation results 
could- be used to prove the efficacy of the project and justify its contin- 
uatiorv. Where the resTults did not demonstrate successful accomplishment of 
objectives, either retooling or discontinuation of the project could be 
demanded through th'e use of objective evidence. Many of the original Title I 
projects in Cleveland have been discontinued or modif ied'because of the > 
evaluations which were done. Some have been shifted to other funding 
sources after modification. 

This usage of ev„aluation just described has followed some very specific 
procedures and guidelines in the Cleveland system. l-Kien a project is initial-ly 
written, it is qftentimes done by persons who efre not the eventual imple- ' 
mentors of .the project. It may be that the objectives and procedures are ' 

found to be inappropos once the project is initiated. The evaluation will 

\ r . ■ » 

US'^illy indicate a non-attainment df certain objectives. I have found it best 
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at this paint to allow the operational people to formulate" the objectives of" 
the project for the secoRd'^year ;of operation. Jn thi^ way they can bear 
more responsibility for the attainment of the objectives, and jiot be able to 
state that the main reason for the failure to meet oJ)j'ect'ives was^ their 
inappropriateness to the actual setting in the first place-. , , ..' 

Therefore., the first evaluation becomes a monitoring'device to ascerta 
v/hether the project is on course. Even though much can be. said about the. 
chang,es 'that' occur in*pupils, I do.not put too much .weight on 'such. findings 
after^ly a year of operation, ^usually turn -to the skorfd year and its 
evaulation as the proof of the pudding. If the 'f ind"i;igs do 'not indicate' " 
attainment of the objectives that have been revised by the project people, 
it is at this point tiiat I begin to question the viability of the project. 
At this point I can ^demand retooling. If future evaluations after the 
second year do not indicate attainment of "project objectives, I'then have ' 
outside objective^ evidence to discontinue either certain activities or the 
project itself.' ^ ^- s 

•i 

' In the case which I havejust described, evaluation is used as a 
management tool to assist decision-making and to reallocate resources when 
results indicate thaf^bject.ives are not being met'. 

But^things do not a.lways work out iri'such a Utopian way. - It i's a ' 

i i 

fallacy to assume that if the truth i'^ known it will 'be acted upon.^Inherent 

conservatism' and a need to niaintaih the statlis quo are ofte^'more important 

than the results of an objective analysis that shows a" nged for an/aijterna- 

• ' • . . > ' 

tive course of action; Educators reject rapid change tha.l; upsets- the status 

quo despit&^its educational value. Evaluation is todoften not used by thosd 



far whom it- is intended because its results challenge current practice. 
Evaluate must therefore balance the need to maintaia'the status quo with 
the need for change or reform. ^ . . ' . ' • ^ 

Accentuate Something Positive ^/ ' * . , 

' Ks a user of these evaluation result^I would I'ike to make' certain 
observations arid recommendations 'conceronng the evaluation of^ Title I as I 
have seen it and as I would like/ to see it. * . . / 

When I was engaged in classroom teacjiing'^ I femember:. one of the prjnci- 
palXcontinuously reminiJing us to. "say„someth1iig" good about t+ie, child" when- 
ever we talked, to parents or wrote comments on'repb^^t 6ards. iFfiis was sound 
advice then and it is sound advice for ,eva1aa tors qI; today's projects. ■ ' 
Evaluatic/h tfiat is in some way supportive of school system policy .is usually' 
considered successful.. I'have to b'efieve that people who work in special / 
projects: do want "to see the project succeed.. For this they shou'ld, see 
something stated positively about what they^ave done/.even if the over-all . 
objectives of the" project are' not met. Of all the national studies of ESEA 
Title I that have been done in'the past decad^j,' the ones which received the 
most pOblicity were 'Ehe American Institute -pf, Research (AIrV studies ;. their . 
series was entitled '.'It Work-s" an^oncentrated eui a'f§w successful compen- , 
satory projects." T-hey did not "publ^jsh or chal lenge- what did not work, - 
although matiy knew that hundreds of .prb'jects wer^ rejected because they'did 

not meet>the stringent cRite'r-ia needed for inclusion- inthe final repbrt.- 

j ■ ' '\ ■ ' " 

It may very w^ll be.-that legislators seem'more concerned about' the"- . , 

distribution pT Title I furfds than they are about the eT/ects'of the funding. 



but it roust be r'emembered^ tha-t, Congress*v/ants to be. seen, as , wise arid' jud.iVimjs • 
" . • *■ > . . , , . ' 

'and wants to see the success of .its initiative? tov^rd Tit! e. I ' Congressmen 

welcome reports of successfuTTitle I progra/ns in their states or district;. '•. 

they want to hear success stories.. However, they^ haVe not had too many . 

success stories to tell,,^ { am .convinced 'that Title I has' dpne some gopd but 

^ , that we have not done ^ough to Accentuate the pb^iti've. As a rqsuH,/it 

'has become more cdmmoriplace to say that' "Title I is not working." 

Ijteed For Consistency In Reportirfg Result's ' ' • 

' ■ , ■ . 

Some of thg-data which is collected Should be able to meet both local and 

national needs Tor information concerning the^results o'f Title I. There is a 

-,'»..■•- ■ ■* - 

• need- "for suctiigons'ls'tency. so th^t unified statements about;' the 'effects of 

V Titl6 I can.be ma^e. such methods, input can -be made to Congress. 'In » 

. identifying what kinds of data shotftld b'e collected; in order to Wed the needs • 

^"^v^locaT evaluation and the needs qf Congress 'we must also consider 'what .is • 

^^^^ _ , ' ^ , 

/most ae^^sible and most easily understood/ . * * *^ 

A .commoR ""Strand of effectiveness shoul.d be. included ir? each and every ' 
evaluation. Qf a TittK| project, -^^ducaVors differ "as to what 'constitutes a 
sucCe^sfvH Title .1 projetl^^^feut it- getl db'wn to a. c-hild' s achieving some " • ' " 
proficiency i-n basic ^kilT^, thaVlji^d. no\ have previously.. At "present, " 
ethere are no uriiformTy accepted-criterfa'.fpAdetermi'ning what constitutes a* . • 
successful- l)rogram/a Ithoagh' MCE ' s ' (nlmal cuWe>^ui va 1 ent/) -are now 



being proposed". , . 

' '' , ^ • • ■ _ ^ ^ ■ 

•The recent GAO report on ESEA Title\ I indicated tKat the Of f ice" of 
•Education..has not Required adequate inforr^ation' from states -^and' local school 



- districts for measuring the n^t^Tonal impact of Title I projects on impt'ovfjig 
_ the achievement' of pupils/^^e -GAO investigation found that local evalua- • 
ti'ons, wlitch are^-the basjs of state evaluations to USOE, generally lack 
un-iform datayfind have often been incomplete. 

From my experience, even local information seekers continue to request ' 
data in terms of the achievement made by pup,ils in the basic ski.lls of reading 
and mathematics. Therefore, such data should be included i/any reports madd 
.. about Title I. I propose that achievement re$ults.i;i basic skills be the 
common strand throughput evaluation reports. Other data needed concerning 
pupil, characteristics or the attitudes of both pupils and parents can be 
made as a supplement to this common data concerning reading and'mathematiis " 
gains. Evaluations that neglect to include data concerning' academic achieve- 
ment usually fail to give f ederal'program directors the kind of concrete data. , 

• they rie-ed ta defend Title I expenditures to a highly C.ritical questioner." 

, As mentioned previously, program ditec tors fear threatening evaluations." 
AThey do- want to improve programs or even replace them, but bec&use of Ipeff 
pressufes to continUe-Qertairt thtrenched programs, they want to'be^ to ' 
fave.face ;n doing, so. In' thisjame vein, there is-also the -fear-tjiat persons 
in the community wi/iruse ahy negative evaTuatiorrfto attacjc- the schools.:-. 
Therefore, informati-oh which is not^ongru^ttt^with the- self iinterests of 'the 
system has a good chance of being i'^nored or suppressed - 

. Xayman's L-antjuage ' '. • . , 

' . • : ■ ' ^ ■ - . ' 

• .■ . Evaluators sfipald continuously bear in mind that one of -the responsibilities 
'imposed on school systems^ is dissemination of inform ,,on about the operation of 
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Title I. Since the' evaluation report itself is a majpr vehicle used in, 
dissemination, the information should' be- presented in ways.'that are simple 
for the .interested party to read and comprehend. The language of the .narra- 
•Vive "should be in layman's language; it should'al'so be comprehensi ble to ' 
parents who in their advisory ^cb1kit;t€e roles have become a .party to the; 
operation of Title I. It V/ou^-d seeii that the best ^ay to,acco"tnplish this 
-would be to have the valuation divided into both a technical section and a 
summary section. The technical section- could 'be for those persons whp want 
in-depth information; -the summa.ry section could be written in- non-technical 
language and couTd ^ej^ve the purpose of either the parent or the person who 
is looking for^^a^general overview of the .results. , ' 

One of the original hopes of ESEA Title I Was -that promising, practices 
would be expanded-and replicated' elsewhere./- However,., it has now become 
evident that a multiplicity of " influences govern -local district, poT icy , and 
success, in one place does not necessarily guarantee succes's in another pl-ace. 
Itv^must be understood' that dissemination df information about successful' 
practices does not lead to theii^ a-dpption elsewhere or even to their expan- 
sion in the same district.* Nevertheless , such a consfderation should not 
preclude the. locaT district hiaving a formal sys.tem foi^. 'disseminating, infqr- 
mation about its exemplary activities to an* thos« concerned/*^ i . ~ 

Frequency of Evaluations \ , . ' \ - ^ ' ' ^ • ^ • 

"Another problem that I see is the viewpoint tliat particif'l£!^P^atiivities 
must be evaluated each and .every year; much tim^ and effor^Heems to be ' - ' 
duplicated every year in preparing a .new evaluation report. 'As^e reads 



tJbe evaluations of a phojfect in successive years, there seems to be. much ' 
duplication from one year to' the next. P^aps it would be possible to 
evaluate- objectives Concerning gains in academW^^hievemejit eaOi year, and 
do some of the supplementary items only every few years . The AIR studies ' 
did show that there are inconsistencies from year to year in a particular 
project, but I have not found significant enouqh differences to warrant a 

, thorough evaluation of all aspects'of a project every year. The one thing 
that I have found to be tlie single most impot*tant factor in making a project 
successful is peop\e. I can always remember .someone who said that there is 

'magic in, people and not' in things. As long ai the staff of a project remains 
essentially the same from 'one yoar 'to another, as long as at least tv/o suc- 
cessive evaluations have shov/n positive res^l,ts, and as long as^ the objectives 
and pV'ocedures of a project remain the same, I dp not see how the evaluation 
woul,d change too much iii a year's time.^ As a result, annual extensive 
evaluations of every aspect of a project would seem unnecessary. 

lonqituding! Studies ^ ^ ^ 

, Some evaluations contain a section/ on sior^e type of longitudinal study 
that has been done'orf some past participants in Title I projlfets. I feel 
'that many of these studies' are not well -founded are misunder-itood, and as a 
result should probably not be included in evaljuatiori reports. Let me hasten' 
to add that I arn^ well-aware that the latest. GAO report on thj operation of 
Title I in certain selected states found longitudinal studies on former 
Titl'e I pupi'ls to, be Seriously lack-ing. I would favor snbh longitudinal 
studies only on the condition thA they are iiifoirr;' seeking and are not 
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used to evaluate the ov.er-all effectiveness' of Title I: The premise of such 
^ studies would'have to, be that once a pupil has been treated in a- Title I 
project he should be able to be placed ba.ck into the mainstream and proceed 
as a normal child without further problems. . ^ch a phenomenon is not ' 
expected of the medical profassion, arid I do not feeV that such ,a procedure 
should be expected of the education profession. When a patient goes to'a 
physician with a strep throat and is given a prescription that^ clears up the 
malady, it is not expected that the strep throat will not reoccur, several 
months later or next year. Likewise, when a child with a reading* deficiency 
goes to a remedial reading teacher and receives prescriptive treatment, (t 
should not be expected that the problem will not reoccur in the future or.ce 
the special- treatment is stopped. The.author realizes that one situation 
involves a physical condition whereas the, second situation involves an- - 
intellectual condition, and that the comparison may be considered too 
simplistic. Too often longitudinal studies are done with the hope that 
Title I services to a pupil will cure his malady forever., *' 

I firmly believe that such a child wouTJ'havp to recelve^continuing 
support if he is to progress at a normal rate. Title I service is designed"" 
to allow the ch^ld t^ose the gap; it\i4 not ainTed at c>ianging his whole 
learning pattern. 

. ' Usually the results of such longitudinal studies are negative, and . 
conclusions are reached that the treatment was not so good after all. I 
personally feel that the RAO was incorredt in looking for longitudinal- results 
on individual pupils and not differentiating between cl osing the gap and 
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changing a learning pattern. Besides, expenditure of Title I, funds on. 
longitudinal studies of pupils no-longer in a Title I project wquld be a 
questionable TitTe I expense- since these pupils are no longer eligible Yor ' 
Title I ser.vlces or expenditures. 

Tail (fagginn The- Dog? 

Last:of an, I feel that we must be careful to assure^ that evalua^on 
does not become the proverbiaVtail that wags the dog. Many times- I have 
found that a particular operational procedure is questionable because it • 
may contaminate tfie eval-uation ■&r-1-ea4-4;0'a situation where a pupil ^will 
receive two treatments with the result that the evaluation vJill not be able 
to dikinqui,sh what effect either treatment "had on the resulfs. As a person 
who IS responsible for assuring that pupils receive needed ^services, I feel 
that the pupil must be given every opportunity to obtain all such services, 
even as the expense of other things such as evaluation. Conflicts can arise 

over such items as late errtry into a"project, exclusion of a child from 

ftp 

services so that he may serve as part of a control group, or receipt of more 
than one treatment. J wouSd recommend that when a CQjiflict arises beWeen 
service to a child and evaluating procedures , ^that particular. child be 
excluded' from the evaluation for that project. If necessary, the progress 
of such childr'en can be included in a separate section or in <t .grand 
composite for the. project. Evaluation should not be allowud to be the main 
reason why a particular service is given or not given t-i a child. 

In, conclusion , I make a plea for unity. He mu^+ continue to enphas^e. 
that making progress in the teaching of the basic ills of readina and ' 



mathematics is the responsibility of everyone associated with the operation - 
of ESEA Title I. Every staff member, whether- an evaluated or'a member of the 
service staff, shou/d 'di rect his efforts in this direction'. Everyone sh&uld 
cooperate in "rowings the canoe in the same direction," i-e.,^.,,the direction 
of improved instructional services for pupils. ' 



